A Spatial Approach to Speech and Ges- tural Control in Collaborative Virtual En- vironments

This paper explores issues relating to the use of broadcast media such as speech and gesture for interacting with non-human (i.e. application) objects in collaborative virtual environments. These issues arise from a recognition that broadcast media such as these are inherently social in nature and that, in order to join in such media, an object should possess a degree of social awareness. Put in simple terms, in addition to being smart enough to understand the content of the input they receive, objects which are controlled by speech and gesture must also understand when they are being addressed and, conversely, when utterances they ÒoverhearÓ are intended for others. This issue becomes particularly significant in environments were multiple people simultaneously attempt to control multiple objects. Our paper begins by clarifying the nature of the problem and outlining three examples of possible techniques for resolving it. We then focus on one of these techniques, the use of spatial awareness mechanisms. We explore one specific example of this technique and present two demonstrators that have been implemented in the MASSIVE collaborative virtual reality system.